Computer graphics technology provides methods to generate and process imagery on a computer. Unlike video, which represents realistic scenes from the real world, computer graphics is generated-imagery such as computer game visual imageries, animations, drawings, designs, graphical user interface, all of which are authored by graphics creation tools, and the like. With the advancements in computer graphics technology, computer graphics have evolved from two dimensional (2D) graphics such as icons, menu, pictures and other 2D objects, to moving images (i.e. animations) and to 3D graphics with 3D perspectives.
As computer graphics technology has improved, 3D computer graphics have become widespread today. In contrast to 2D computer graphics technology, 3D computer graphics technology use 3D graphics data (e.g. overlay mapping data, geometric data in a 3D Cartesian system, etc.) for the purposes of performing calculations and rendering 3D imagery on a computer. Various techniques in 3D computer graphics technology have been developed to achieve this purpose. One category of such techniques overlays 3D graphics data, such as surface texture, bumps, displacement, etc, on top of coarse geometry models to add a perception of detailed depth (e.g., skinning as generally known in the art). Such overlaid 3D graphics data is commonly defined in image maps as known in the art and in itself also forms images, e.g. raster images.
For example, texture mapping is a 3D graphics technique that employs data defining surface textures (e.g. color, lighting, shadow, or anything that conveys detailed texture information about the graphics) to add significant amount of details that simulate the appearance of materials. Typically, texture mapping applies a collection of texture data, i.e. a texture map as known in the art, to a surface of a shape or a polygon in an image every time when the surface is rendered. A light map is an example of such texture maps, wherein the light map comprises data defining lighting information for every texture coordinate corresponding to the surface. As such, the light map can be used to “light” the surface by applying (i.e. mapping) the lighting information to the surface every time when the surface is rendered.
As an improvement, normal mapping was developed to add effects based on surface normal data (e.g., imaginary lighting that is applied perpendicular to a surface of a polygon in an image) to increase lighting effects generating the perception of bumps and dents created by the reflection and/or refraction of the surface normals. Normal maps typically comprise data that defines normal vectors, which are encoded by colors. Normal maps use three channels (3 bytes) to encode this information to add additional lighting effects to the surface. With this information, normal mapping can create an illusion of high resolution on what is actually a low resolution surface.
Displacement mapping is an alternative to normal mapping in 3D graphics that adds the perception of details to a surface through geometry alteration. Instead of adding surface normal effects to the lighting of the surface, displacement mapping employs geometric data to alter the geometry of the surface, e.g. displacing certain points of the surface to new positions. In so displacing, normal mapping gives the surface a great sense of depth and detail, permitting self-occlusion, self-shadowing and silhouettes, etc to be shown in the graphics.
The image maps used by 3D graphics techniques, e.g., the texture, displacement, normal maps, etc, form one or more images, etc.
In addition to generating 3D effects using various mapping techniques described above, 3D computer graphics technology also provides various methods to generate 3D graphics by employing data that defines 3D geometric models, i.e. the shape, for an object. These models describe 3D objects on a computer using mathematic primitives such as spheres, cube, cones and polygons, etc. The most common type of 3D geometric models is polygon mesh, also known as 3D mesh. 3D mesh is represented by a collection data of vertices, edges, faces, and polygons, etc defining the shape of a 3D object. With such data, a render on a computer graphics processing system can generate a 3D representation of the object in polygonal forms as an approximation of the object. Accordingly, the more curved a 3D object's shape is, the more data is needed to define polygons constituting the object for an accurate approximation. In any event, data defining 3D a geometric model like 3D mesh usually contain many polygons, triangles and other necessary shapes interconnected to one another.
Furthermore, developments in 3D graphics technology have taken 3D meshes defined as static to the next step by providing methods that generate animation for 3D meshes. For example, skeleton-based animation technique has been developed to employ 3D mesh data defining a skeleton (i.e. the skin and a hierarchical set of interconnected bones of a 3D character in a computer game) and as well as data defining movements of the 3D character. As such, this technique, for example, animates the 3D character by moving the bones of the skeleton in an articulated manner. Frame-based animation is another 3D graphics technique that employs data to define 3D meshes and their movements over time. Unlike skeleton-based animation, frame-based animation generates animated 3D meshes through defining movement changes in time, i.e. frame by frame, rather than in space.
For all of the aforementioned 3D graphics techniques, large amount of information to facilitate relevant representation of 3D content on a computer is required. Today, computer games, computer graphical designs, drawings, medical imaging, computer cartoons and the like have increasingly utilized 3D graphics in a combination of various forms described above to represent complex 3D objects on a computer. These objects are often defined with extensive texture information on top of 3D geometry models like meshes that have a tremendous amount of vertex points defining the shape of the 3D objects to the finest degree possible. In the case of animated 3D objects, the amount of graphics data is significantly increased even more for defining movements of the 3D objects. Often, the size of graphics data representing 3D content in a 3D graphics application like a computer game is on an order of several magnitudes larger than graphics data representing conventional 2D content. As a result, problems of large storage needed to store such data on a computer and long transmission time for transmitting such data over computer networks have emerged.
An effective solution to the problems of large storage requirements and long transmission time associated with 3D graphics data is using compression techniques to compress 3D graphics data into smaller size. Various compression methods compressing 3D graphics data have been proposed. For example, variable bit rate (VBR) compression, e.g., using JPEG, JPEG2000, and PNG, etc. has been employed to compress 3D graphics data using a variable compression ratio on a per block level. The advantage of such VBR compression of 3D graphics data is that the VBR compression adapts to data such that every block of the 3D graphics data is compressed according to its own condition using a variable ratio and thereby it compresses the 3D graphics data very efficiently. However, decoding VBR compressed 3D graphics data requires processing at a software level performed on a host central processing unit (CPU). This is because existing specialized hardware graphics subsystem on a computer (e.g. GPU acceleration hardware) currently does not support VBR decoding, and hence software decoders must be employed to decode VBR compressed 3D graphics data. It is generally known that decoding at the CPU level through software decoders is not as efficient as hardware decoding in terms of power consumption due to various overheads associated with execution of software decoders. As a result, decoding VBR compressed 3D graphics data at the CPU level using software decoders consumes a lot of power. This power consumption problem is particularly acute to laptop computers, tablets, smart phones, or other portable devices whose main power source is battery on the go. Another problem associated with software decoding of VBR compressed 3D graphics data is the extra data path between a CPU and GPU, where 3D graphics data as decompressed by software decoders must be transferred from CPU to GPU for further processing. This creates extra data path from the CPU to GPU compared to when graphics data is directly loaded into GPU for processing without being decoded at CPU level. In view of 3D graphics data typically having large data size after decompressed by the CPU, this adds another source of power consumption and transmission delay.
As an improvement, GPU friendly compression like the DXTn standards have been developed to facilitate faster and more efficient of processing of 3D graphics data at the GPU level. DXTn eliminates the data path between a CPU and GPU by enabling the GPU to directly process compressed 3D graphics data, known as “on-the-fly” processing in the art. The most recent DXTn standard is DXT5 and it can compress each 4×4 RGBA block to 128 bits for a 4:1 compression ratio. Accordingly, compression using DXT5 and as well as other DXTn standards is fix bit rate compression methods in that it invariably compresses each 4×4 block of graphics data with the same 4:1 compression ratio. As a result, while improving over software decoders in terms of efficient processing and relatively low power consumption, DXTn compression, as a fix bit rate compression method, inevitably contains some blocks that are over-compressed leading to artifacts while some other blocks are under-compressed.
As an improvement directed towards the under-compression problem of DXTn, further encoding DXT compressed 3D graphics data using entropy encoding. Entropy encoding is a well known encoding method that basically replaces each source symbol with the smallest number of bits (i.e. entropy). However, entropy encoded data can also only be decoded at software level on a CPU. Accordingly, the afro-mentioned problems of power consumption and transmission delay at CPU level are also associated with entropy encoding of DXT formatted 3D graphics data.
Recent efforts in graphics processing have focused on developing specialized hardware to enable a GPU to accelerate video decoding or encoding. For example, existing specialized GPU video acceleration hardware implements video codecs, such as H.264, with blocks of circuitry logic and thereby encode/decode data of video more efficiently by off-loading such operations from the CPU to GPU to achieve accelerated video encoding and decoding. One of corollary benefits associated with hardware accelerated video encoding/decoding, compared to software video decoder/encoder running the CPU, is that significant power consumption is saved since the specialized acceleration hardware encodes and decodes videos more efficiently without incurring typical overheads unavoidably associated with software video decoder/encoder when executed on the CPU. Another benefit of hardware video acceleration is reduction of bandwidth by skipping the CPU processing of the video entirely—i.e. the video is directly loaded onto the GPU for processing without being loaded to the CPU and then from the CPU to the GPU. However, video acceleration hardware is not designed to process 3D graphics data since 3D graphics data does not define video. Accordingly, the efficiency of existing GPU video acceleration hardware has not been applied to the processing of 3D graphics data.
Therefore, for one or more above-mentioned problems, there exists a need for a method and apparatus for enhanced processing of 3D graphics data.